Management system for processing streaming data

ABSTRACT

Techniques are disclosed for evaluating tuples for processing by a stream application having a plurality of process elements. In one embodiment, at least one tuple to be processed by at least one processing element of the stream application is identified. A maximum duration for which the at least one processing element is allowed to process the at least one tuple is determined. A duration for which the at least one processing element is likely to process the at least one tuple is also estimated. Processing of the at least one tuple is managed based on a comparison between the maximum duration and the estimated duration.

BACKGROUND

Databases are computerized information storage and retrieval systems. Arelational database management system is a computer database managementsystem (DBMS) that uses relational techniques for storing and retrievingdata. An object-oriented programming database is a database that iscongruent with the data defined in object classes and subclasses.

Regardless of the particular architecture, a requesting entity (e.g., anapplication or the operating system) in a DBMS requests access to aspecified database by issuing a database access request. Such requestsmay include, for instance, simple catalog lookup requests ortransactions and combinations of transactions that operate to read,change and add specified records in the database. These requests areoften made using high-level query languages such as Structured QueryLanguage (SQL). Upon receiving such a request, the DBMS may execute therequest against a corresponding database, and return any result of theexecution to the requesting entity.

SUMMARY

Embodiments of the invention provide a method, system and computerprogram product for performing an operation that includes receivingstreaming data to be processed by a stream application comprising aplurality of processing elements. Each processing element includes oneor more operators that are contained within a single process. Thestreaming data includes a plurality of tuples. The operation alsoincludes identifying at least one tuple to be processed by at least oneprocessing element of the stream application. The operation alsoincludes determining a maximum duration for which the at least oneprocessing element is allowed to process the at least one tuple. Theoperation also includes determining an estimated duration for which theat least one processing element is likely to process the at least onetuple. The operation also includes managing processing of the at leastone tuple, based on a comparison between the maximum duration and theestimated duration.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited aspects are attained andcan be understood in detail, a more particular description ofembodiments of the invention, briefly summarized above, may be had byreference to the appended drawings.

It is to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIGS. 1A-1B illustrate a computing infrastructure 100 configured toexecute a stream application, according to one embodiment of theinvention.

FIG. 2 is a more detailed view of the compute node 130 of FIGS. 1A-1B,according to one embodiment of the invention.

FIG. 3 is a more detailed view of the server computing system 105 ofFIG. 1, according to one embodiment of the invention.

FIG. 4 illustrates an example of compute nodes in a stream applicationwhich create a wait condition, according to one embodiment of theinvention.

FIG. 5 illustrates an example of compute nodes in a stream application,according to one embodiment of the invention.

FIGS. 6A-6C are bar graphs depicting estimated processing durations oftuples, according to one embodiment of the invention.

FIG. 7 is a flowchart depicting a method for evaluating a tuple forprocessing, according to one embodiment of the invention.

FIG. 8 is a flowchart depicting a method for managing processing of thetuple, according to one embodiment of the invention.

DETAILED DESCRIPTION

Stream-based computing and stream-based database computing are emergingas a developing technology for database systems. With this emergingtechnology, users can create applications that process and querystreaming data before it reaches a database file. Further, users canspecify processing logic to apply to inbound data records while they are“in flight,” with the results available in a very short amount of time,often in milliseconds. Constructing an application using this type ofprocessing has opened up a new programming paradigm that will allow fora broad variety of innovative applications, systems and processes to bedeveloped, as well as present new challenges for application programmersand database developers.

In a stream application, data may be continuously read and passed fromone processing element to another. A processing element refers to acollection of operators that are contained within a single process. Anoperator refers to an executable piece of code that is connected toother operators, such that data flows from one processing element to thenext (e.g., over a TCP/IP socket). Scalability is reached bydistributing an application across nodes by creating many smallexecutable pieces of code (i.e., the operators), as well as replicatingprocessing elements on multiple nodes and load balancing among them.Processing elements (and operators) in a stream application can be fusedtogether to form a larger processing element. Doing so allows processingelements to share a common process space, resulting in much fastercommunication between operators than is available using inter-processcommunication techniques (e.g., using a TCP/IP socket). Further,processing elements can be inserted or removed dynamically from anoperator graph representing the flow of data through the streamapplication, as well as fused or un-fused from a stream applicationduring runtime. The operator graph may be used to divide and/ordistribute a workload across the processing elements, so that data maybe processed with increased parallelism.

One advantage of stream applications is that they allow the user togranularly control the process flow of data through the application. Inother words, the user may designate specific operators for eachprocessing element that perform various operations on the incoming data,and may dynamically alter the stream application by modifying theoperators and the order in which they are performed. Additionally,stream applications are able to handle large volumes of data whilelimiting any “bottlenecks” in the processing. Further, streamapplications may meet the performance requirements of some real-timeapplications, where database systems do not. Database systems aretypically configured to separate the process of storing data fromaccessing, manipulating or using data stored in the database. Morespecifically, databases use a model where data is first stored, thenindexed, and finally queried. Accordingly, the rate at which a databasesystem can receive and store incoming data limits how much data can beprocessed or otherwise evaluated. Advantageously, at least in somecases, the ability of stream applications to process large amounts ofdata in real-time is not necessarily as limited as compared to databaseapplications.

Embodiments of the invention provide techniques for managing processingof streaming data. The streaming data may include N-tuples of dataattributes. One embodiment provides a stream pacer configured to receivea tuple to be processed by a first processing element. The stream pacerdetermines a maximum duration for which the first processing element isallowed to process the tuple. The maximum duration may be determinedbased on real-time indicators of workload experienced by the firstprocessing element and/or the streaming application. One example of areal-time indicator is a buffer level for inbound tuples, whichcharacterizes a degree to which the buffer is full from storing tuplesthat remain to be processed. The stored tuples may be received from oneor more upstream processing elements. Additionally or alternatively, themaximum duration may be user-specified and tailored to suit the needs ofa particular case and/or processing element. The stream pacer alsodetermines an estimated duration, where the estimated duration refers toa duration for which the first processing element is likely to processthe tuple. The estimated duration may be determined based on historicaldata collected from processing of previous tuples by the firstprocessing element. Depending on the embodiment, the estimated durationmay vary based on the processing element and/or one or more dataattributes stored in a tuple.

In one embodiment, the stream pacer then manages processing of thetuple, based on a comparison between the maximum duration and theestimated duration. For example, if the estimated duration exceeds themaximum duration, the stream pacer may reject the tuple from beingprocessed by the first processing element. The stream pacer may alsosend a message to a requesting entity, explaining that the tuple wasdenied for processing. In some embodiments, the stream pacer may repeatone or more of the above steps to determine whether to approve the tupleto be processed by a second processing element. On the other hand, ifthe estimated duration does not exceed the maximum duration, then thestream pacer may approve the tuple for processing by the firstprocessing element. In some embodiments, the stream pacer may also haltthe processing of a tuple that is already being processed, if theestimated time remaining for the processing of the tuple to completeexceeds the maximum duration. However, it may not always be desirable tohalt the processing of tuples being processed. Accordingly, in oneembodiment, a user may set a flag that specifies not to halt tupleprocessing under any circumstances. Thus, the behavior of the streampacer in halting tuple processing may be overridden to suit the needs ofa particular case.

Advantageously, the stream pacer may manage the processing of streamingdata in a manner that is more responsive to the needs of the processingelements and/or users at least in some cases. For example, the streampacer may impose a maximum duration for tuple processing, which mayreduce a likelihood of a given processing element being overwhelmed fromprocessing an enduring tuple. Accordingly, the number of wait conditionscreated for downstream processing elements may be reduced. In someembodiments, the maximum duration may be specific to a given processingelement and/or data attribute stored in a tuple.

Further, in some scenarios, such as in stock trading and surveillancework, data may have no value if not processed within an associatedmaximum timeframe that is user-specified. Accordingly, the stream pacermay reject a tuple from being processed if the stream pacer estimatesthat the tuple is not likely to be processed within the maximumtimeframe. In addition, the stream pacer may allow fewer tuples to beprocessed by a given processing element, when the processing element isexperiencing a heavy workload. On the other hand, the stream pacer mayallow more tuples to be processed by the given processing element, whenthe processing element is experiencing a light workload.

In the following, reference is made to embodiments of the invention.However, it should be understood that the invention is not limited tospecific described embodiments. Instead, any combination of thefollowing features and elements, whether related to differentembodiments or not, is contemplated to implement and practice theinvention. Furthermore, although embodiments of the invention mayachieve advantages over other possible solutions and/or over the priorart, whether or not a particular advantage is achieved by a givenembodiment is not limiting of the invention. Thus, the followingaspects, features, embodiments and advantages are merely illustrativeand are not considered elements or limitations of the appended claimsexcept where explicitly recited in a claim(s). Likewise, reference to“the invention” shall not be construed as a generalization of anyinventive subject matter disclosed herein and shall not be considered tobe an element or limitation of the appended claims except whereexplicitly recited in a claim(s).

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Embodiments of the invention may be provided to end users through acloud computing infrastructure. Cloud computing generally refers to theprovision of scalable computing resources as a service over a network.More formally, cloud computing may be defined as a computing capabilitythat provides an abstraction between the computing resource and itsunderlying technical architecture (e.g., servers, storage, networks),enabling convenient, on-demand network access to a shared pool ofconfigurable computing resources that can be rapidly provisioned andreleased with minimal management effort or service provider interaction.Thus, cloud computing allows a user to access virtual computingresources (e.g., storage, data, applications, and even completevirtualized computing systems) in “the cloud,” without regard for theunderlying physical systems (or locations of those systems) used toprovide the computing resources.

Typically, cloud computing resources are provided to a user on apay-per-use basis, where users are charged only for the computingresources actually used (e.g., an amount of storage space used by a useror a number of virtualized systems instantiated by the user). A user canaccess any of the resources that reside in the cloud at any time, andfrom anywhere across the Internet. In context of the present invention,a user may access applications or related data available in the cloud.For example, the nodes used to create a stream application may bevirtual machines hosted by a cloud service provider. Doing so allows auser to access this information from any computing system attached to anetwork connected to the cloud (e.g., the Internet).

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality and operation of possible implementations ofsystems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

FIGS. 1A-1B illustrate a computing infrastructure configured to executea stream application, according to one embodiment of the invention. Asshown, the computing infrastructure 100 includes a management system 105and a plurality of compute nodes 130 ₁₋₄, each connected to acommunications network 120. Also, the management system 105 includes anoperator graph 132, a stream manager 134, and a stream pacer 136. Asdescribed in greater detail below, the operator graph 132 represents astream application beginning from one or more source processing elements(PEs) through to one or more sink PEs. This flow from source to sink isalso generally referred to herein as an execution path. Generally, dataattributes flow into a source PE of a stream application and areprocessed by that PE. Typically, processing elements receive an N-tupleof data attributes from the stream as well as emit an N-tuple of dataattributes into the stream (except for a sink PE where the streamterminates). Of course, the N-tuple received by a processing elementneed not be the same N-tuple sent downstream. Additionally, theprocessing elements could be configured to receive or emit data informats other than an N-tuple (e.g., the processing elements couldexchange data marked up as XML documents). Furthermore, each processingelement may be configured to carry out any form of data processingfunctions on the received tuple, including, for example, writing todatabase tables or performing other database operations such as datajoins, splits, reads, etc., as well as performing other data analyticfunctions or operations.

In one embodiment, the stream manager 134 is configured to monitor astream application running on the compute nodes 130 ₁₋₄, as well as tochange the structure of the operator graph 132. The stream manager 134may move processing elements (PEs) from one compute node 130 to another,for example, to manage the processing loads of the compute nodes 130 inthe computing infrastructure 100. Further, stream manager 134 maycontrol the stream application by inserting, removing, fusing,un-fusing, or otherwise modifying the processing elements (or whatdata-tuples flow to the processing elements) running on the computenodes 130 ₁₋₄.

In one embodiment, the stream pacer 136 is configured to manageprocessing of a data stream. More specifically, the stream pacer 136 mayevaluate a tuple to determine whether to approve (or reject) the tuplefor execution by a given processing element. The determination may bemade based on a comparison between a maximum duration and an estimatedduration for the tuple and/or processing element. At least in someembodiments, some or all of the functionality of the stream pacer 136may be performed by the stream manager 134. For example, the streampacer 136 may be implemented as a stream pacer component of the streammanager 134.

In one embodiment of the invention, the management system 105 maymaintain multiple operator graphs 132. In such an embodiment, oneoperator graph 132 is designated as the operator graph, which representsthe general or default processing flow, while the other operator graphsmay represent alternate processing flows.

FIG. 1B illustrates an example operator graph that includes tenprocessing elements (labeled as PE1-PE10) running on the compute nodes130 ₁₋₄. While a processing element may be executed as an independentlyrunning process with its own process ID (PID) and memory space, multipleprocessing elements may also be fused to run as single process (with aPID and memory space). In cases where two (or more) processing elementsare running independently, inter-process communication may occur using anetwork socket (e.g., a TCP/IP socket). However, when processes arefused together, the fused processing elements can use more rapidcommunication techniques for passing N-tuples (or other data) amongprocessing elements (and operators in each processing element).

As shown, the operator graph begins at a source 135 (that flows into theprocessing element labeled PE1) and ends at sink 140 ₁₋₂ (that flowsfrom the processing elements labeled as PE6 and PE10). Compute node 130₁ includes the processing elements PE1, PE2 and PE3. Source 135 flowsinto the processing element PE1, which in turn emits tuples that arereceived by PE2 and PE3. For example, PE1 may split data attributesreceived in a tuple and pass some data attributes to PE2, while passingother data attributes to PE3. Data that flows to PE2 is processed by theoperators contained in PE2, and the resulting tuples are then emitted toPE4 on compute node 130 ₂. Likewise, the data tuples emitted by PE4 flowto sink PE6 140 ₁. Similarly, data tuples flowing from PE3 to PE5 alsoreach sink PE6 140 ₁. Thus, in addition to being a sink for this exampleoperator graph, PE6 could be configured to perform a join operation,combining tuples received from PE4 and PE5. This example operator graphalso shows data tuples flowing from PE3 to PE7 on compute node 130 ₃,which itself shows data tuples flowing to PE8 and looping back to PE7.Data tuples emitted from PE8 flow to PE9 on compute node 130 ₄, which inturn emits tuples to be processed by sink PE10 140 ₂.

Furthermore, although embodiments of the present invention are describedwithin the context of a stream application, this is not the only contextrelevant to the present disclosure. Instead, such a description iswithout limitation and is for illustrative purposes only. One ofordinary skill in the art will recognize that embodiments of the presentinvention may be configured to operate with any computer system orapplication capable of performing the functions described herein. Forexample, embodiments of the invention may be configured to operate in aclustered environment with a standard database processing application.More generally, the techniques disclosed herein may be applied to anyenvironment that maintains one or more operator graphs of processingelements, including both streaming and non-streaming environments.

FIG. 2 is a more detailed view of the compute node 130 of FIGS. 1A-1B,according to one embodiment of the invention. As shown, the compute node130 includes, without limitation, at least one central processing unit(CPU) 205, a network interface 215, an interconnect 220, a memory 225,and storage 230. The compute node 130 may also include an I/O devicesinterface 210 used to connect I/O devices 212 (e.g., keyboard, displayand mouse devices) to the compute node 130.

Each CPU 205 retrieves and executes programming instructions stored inthe memory 225. Similarly, the CPU 205 stores and retrieves applicationdata residing in the memory 225. The interconnect 220 is used totransmit programming instructions and application data between each CPU205, I/O devices interface 210, storage 230, network interface 215, andmemory 225. CPU 205 is included to be representative of a single CPU,multiple CPUs, a single CPU having multiple processing cores, and thelike. The memory 225 is generally included to be representative of arandom access memory. Storage 230, such as a hard disk drive, solidstate device (SSD), or flash memory storage drive, may storenon-volatile data.

In this example, the memory 225 includes a plurality of processingelements 235, buffered stream monitor 245 and stream connection data255. Each PE 235 includes a collection of operators 240. As noted above,each operator 240 may provide a small chunk of executable codeconfigured to process data flowing into a processing element (e.g., PE235) and to emit data to other operators 240 in that PE and to otherprocessing elements in the stream application. Such processing elementsmay be on the same compute node 130 or on other compute nodes accessibleover the data communications network 120. The stream connection data 255represents the connections between PEs on compute node 130 (e.g., aTCP/IP socket connection between two separate PEs 235), as well asconnections to other compute nodes 130 with upstream and or downstreamPEs in the stream application, also via TCP/IP sockets (or otherinter-process data communication mechanisms).

As shown, storage 230 contains buffered stream data 260 and historicalinformation 265. The buffered stream data 260 represents a storage spacefor data flowing into the compute node 130 from upstream processingelements (or from a data source for the stream application). Forexample, buffered stream data 260 may include data tuples waiting to beprocessed by one of the PEs 235. Buffered stream data 260 may also storethe results of data processing performed by processing elements 235 thatwill be sent to downstream processing elements. For example, a PE 235may have to store tuples intended for a downstream PE 245 if that PE 235already has a full buffer. On the other hand, the historical information265 is formulated after running a particular stream application severaltimes. Historical information 265, for example, can be the average ofthe amount of data stored in the buffered stream data 260 during aseries of executions of an application. In sum, the buffered stream data260 is a real-time measurement while the historical information 265 is apredictor of how a PE 235 on a compute node 130 will process tuplesbased on prior stream applications—i.e., whether the PE 235 will createa bottleneck.

Associated with the buffered stream data 260 is the buffered streammonitor 245 which evaluates the amount of data stored in the buffer 260.Typically, if the buffered stream monitor 245 determines that the bufferis nearly full of received buffered stream data 260, this indicates thatthe processing element on that compute node 130 is running behind,thereby creating a bottleneck in the execution path. The buffered streammonitor 245 then reports this latency to the stream manager 134 found onthe management system 105. Conversely, the buffered stream data 260 maybe full because the buffer for downstream PE in the execution path isalready full. In such a case, the buffer stream monitor 245 would notreport that the PE 235 is creating a bottleneck.

FIG. 3 is a more detailed view of the management system 105 of FIG. 1,according to one embodiment of the invention. As shown, the managementsystem 105 includes, without limitation, a central processing unit (CPU)305, a network interface 315, an interconnect 320, a memory 325, andstorage 330. The client system 130 may also include an I/O deviceinterface 310 connecting I/O devices 312 (e.g., keyboard, display andmouse devices) to the management system 105.

Like CPU 205 of FIG. 2, CPU 305 is configured to retrieve and executeprogramming instructions stored in the memory 325 and storage 330.Similarly, the CPU 305 is configured to store and retrieve applicationdata residing in the memory 325 and storage 330. The interconnect 320 isconfigured to move data, such as programming instructions andapplication data, between the CPU 305, I/O devices interface 310,storage unit 330, network interface 305, and memory 325. Like CPU 205,CPU 305 is included to be representative of a single CPU, multiple CPUs,a single CPU having multiple processing cores, and the like. Memory 325is generally included to be representative of a random access memory.The network interface 315 is configured to transmit data via thecommunications network 120. Although shown as a single unit, the storage330 may be a combination of fixed and/or removable storage devices, suchas fixed disc drives, removable memory cards, optical storage, SSD orflash memory devices, network attached storage (NAS), or connections tostorage area-network (SAN) devices.

As shown, the memory 325 stores the stream manager 134 and the streampacer 136. Additionally, the storage 330 includes the operator graph132. The stream manager 134 may generally route tuples received as partof a data stream to PEs 235 to be processed, according to the operatorgraph 132. The stream pacer 136 evaluates each tuple to be processed bya given processing element and determines whether to approve the tuplefor processing by the given processing element. The determination may bemade based on whether the tuple is estimated to exceed a maximumprocessing duration. The estimated duration may be determined based onthe historical information 265 and buffered stream monitor 245 locatedon each compute node 130. The maximum duration may be determined basedon data received from the buffered stream monitor 245 located on eachcompute node 130. For example, if the buffered stream monitor 245 of thePE 235 reports to the stream pacer 136 that the buffer is nearly full ofreceived N-tuples, then the stream pacer 136 may determine that a lowermaximum duration is warranted. Alternatively, the maximum duration maybe a user-specified value. Imposing a maximum duration for tupleprocessing may, at least in some cases, reduce a likelihood of a givenprocessing element being overwhelmed from processing an enduring tuple,especially when the processing element is experiencing a heavy workload.

In one embodiment, because stream applications often deal with largevolumes of data, the processing of which is spread over multipleprocessing elements across multiple compute nodes, this presentsadditional challenges for application programmers and databasedevelopers. One such challenge is identifying bottlenecks that arisebecause of wait conditions. In general, wait conditions are processingelements which receive data from two or more processing elements thatdirectly precede the wait condition in the stream. As an example, aparticular processing element may need data from two separate processingelements before performing a certain operation. If, however, one of theprocessing elements provides data to the particular processing elementat a slower rate than the other processing element, the particularprocessing element is limited by the slowest processing element in theflow of data, creating a bottleneck that may decrease applicationthroughput. At least in some cases, imposing a maximum duration fortuple processing using may reduce the number of wait conditions in thestream.

FIG. 4 illustrates an example of compute nodes in a stream applicationthat create a wait condition, according to one embodiment of theinvention. As shown, compute node 130 ₂ includes three processingelements 235 ₂, 235 ₃ and 235 ₄ (labeled PE1-PE3) while compute node 130₃ contains PE 235 ₅ (PE4). In the depicted example, PE1 receives anN-tuple data stream and emits an N-tuple to PE2 and PE3 on compute node130 ₂. Processing elements PE2 and PE3, in turn, emit N-tuples to PE4 oncompute node 130 ₃. In this example, the PE1 receives a tuple whichincludes attributes <name, department>. PE1 takes this N-tuple,separates the attributes, and generates one set of tuples sent to PE2(<name>) and another set of tuples sent to PE3 (<department>). In turn,PE2 and PE3 perform a database write for each tuple received from PE1and send the tuple to PE4. Once received, PE4 concatenates the tuplesback into the <name, department> format, accesses a third-party webservice, and generates a tuple that is sent further downstream in thestream application.

Continuing the example, PE1 divided the tuple into a <name> tuple and a<department> tuple and sent each tuple to a different processing element235 since both were written to separate databases. This example is atypical method implemented by a distributed system to increasethroughput. However, PE4 cannot move onto the next tuple until it hasreceived both the <name> tuple from PE2 and the <department> tuple fromPE3. Thus, no matter how fast PE2 writes the tuple to the database andrelays the tuple to PE4, PE4 cannot continue until PE3 sends thecorresponding tuple. Thus, FIG. 4 exemplifies a wait condition.

FIG. 5 illustrates an example of compute nodes in a stream applicationthat creates a wait condition, according to one embodiment of theinvention. As shown, compute node 130 ₂ includes three processingelements 235 ₂, 235 ₃ and 235 ₄ (labeled PE1-PE3). In the depictedexample, processing element 235 ₂ receives an N-tuple data stream andemits an N-tuple to processing elements 235 ₃ and 235 ₄ (labeled PE2 andPE3, respectively) on compute node 130 ₃. Further, compute node 130 ₄includes processing element 235 ₆ (PE5) which emits an N-tuple to PE2with the attribute of <salary>. Processing elements PE2 and PE3, inturn, emit N-tuples to processing element 235 ₅ (PE4) on compute node130 ₃. In this example, PE 235 ₂ (labeled PE1), receives a tuple whichincludes attributes <name, department>. PE1 takes this N-tuple,separates the N-tuple, and generates one set of tuples sent to PE2(<name>) and another set of tuples sent to PE3 (<department>). PE2 thenconcatenates the tuple received by PE5 and PE1. In turn, PE2 and PE3perform a database write for each received tuple and send the tuple toPE4. Once received, PE4 concatenates the tuples into the <name,department, salary> format and generates a tuple that is sent furtherdownstream in the stream application.

In this example, PE4 remains a wait condition as shown previously inFIG. 4; however, PE2 is now another wait condition. In one embodiment,PE2 is slower than PE3 since it must write into a database both the nameand salary attributes. Thus, the wait condition found at PE4 could beabated by speeding up the execution of PE2—i.e., overclocking theprocessor associated with the compute node 130 ₂. In another example,however, assume that PE2 can perform its database write as quickly asPE3, but PE5 sends a tuple to PE2 at half the rate of PE1. In that case,PE5 would slow down PE2, and thus, PE4. Overclocking PE2 would notalleviate the wait condition at PE4; however, speeding up the executionof PE5 would. This example illustrates that a processing element thatneeds to be sped up does not necessarily have to be located immediatelyprior to a wait condition.

In another embodiment, the stream manager 134 identifies a hierarchy ofwait conditions, e.g., PE4 is the parent and PE2 is the child. Thus, thestream manager 134 would start at the wait condition closest to the end(PE4) and evaluate the prior PEs 235. If none of these were executingslowly, then the stream manager 134 would move to the next waitcondition (PE2). Continuing the example above, the stream manager 134may then determine that PE5 is creating the bottleneck and actaccordingly.

FIGS. 6A-6C are bar graphs 600 ₁₋₃ depicting estimated processingdurations of tuples, according to one embodiment of the invention.Assume that the stream pacer 136 determines an estimated processingduration of seven hundred milliseconds for a first tuple and fivehundred milliseconds for a second tuple, respectively. Assume that theestimated processing durations pertain to a given processing element 235in the stream application. As shown in FIG. 6A, the bar graph 600 ₁includes a first bar 602 representing the estimated processing durationfor the first tuple and a second bar 604 representing the estimatedprocessing duration for the second tuple.

Suppose that the stream pacer 136 identifies that the given processingelement 235 has a medium buffer level. Suppose also that the streampacer 136 determines, based on the medium buffer level, that the maximumallowed processing duration should be six hundred milliseconds. Themaximum allowed processing duration is represented as a marker 612 inthe bar graphs 600 ₁₋₃. Accordingly, the stream pacer 136 rejects thefirst tuple from being executed, because the estimated processingduration of seven hundred milliseconds exceeds the maximum allowedprocessing duration. On the other hand, the stream pacer 136 allows thesecond tuple to execute, because the estimated processing duration offive hundred milliseconds does not exceed the maximum allowed processingduration. The bar graphs 600 ₁₋₃ include a legend 606 indicating that asolid bar 608 represents an allowed tuple and a shaded bar 610represents a rejected tuple.

Referring now to FIG. 6B, suppose that the processing element 235 has ahigh buffer level instead of a medium buffer level. In one embodiment,the stream pacer 136 determines, based on the high buffer level, thatthe maximum allowed processing duration should be four hundredmilliseconds instead of six hundred milliseconds. The maximum allowedprocessing duration of four hundred milliseconds is represented by themarker 612 in FIG. 6B. Because the estimated processing durations of thetuples—which are seven hundred and five hundred milliseconds,respectively—exceed the maximum allowed processing duration of fourhundred milliseconds, the stream pacer 136 rejects both tuples frombeing processed. Accordingly, lowering the maximum allowed processingduration during times of high system load of a processing element (e.g.,as may be inferred from a high buffer level) may prevent the processingelement from being overwhelmed at least in some cases.

Referring now to FIG. 6C, suppose that the processing element 235 has alow buffer level instead of a medium buffer level. In one embodiment,the stream pacer 136 determines, based on the low buffer level, that themaximum allowed processing duration should be eight hundred millisecondsinstead of six hundred milliseconds. The maximum allowed processingduration of eight hundred milliseconds is represented by the marker 612in FIG. 6C. Because the estimated processing durations of the tuples donot exceed the maximum allowed processing duration of eight hundredmilliseconds, the stream pacer 136 allows both tuples to be processed.

Accordingly, increasing the maximum allowed processing duration duringtimes of low system load (e.g., as may be inferred from a low bufferlevel) may prevent the processing element 235 from being underutilizedat least in some cases. In some embodiments, such as during times ofvery low system load, the stream pacer 136 may allow all tuples toprocessed, regardless of estimated processing durations. For instance,the maximum allowed processing duration may be set to a predefined valuerepresenting infinity. Additionally or alternatively, some or all of thefunctionality of the stream pacer 136 may be disabled in this scenario,such that the stream application no longer incurs the overhead ofdetermining maximum and/or estimated processing durations.

Although embodiments are described herein with reference to exemplarytuple processing durations, those skilled in the art will recognize thatthe techniques herein may be applied other finite resources, such asnumber of processor cycles used in tuple processing, amount of memoryused in tuple processing, amount of storage capacity and/or bandwidthused during tuple processing, and amount of network bandwidth usedduring tuple processing. For example, in an alternative embodiment, themaximum amount of memory that a tuple is allowed to consume isthirty-two megabytes under a medium system load, sixteen megabytes undera high system load, and sixty-four megabytes under a low system load. Insome embodiments, the stream pacer 136 imposes multiple constraints oneach tuple, each constraint corresponding to a different resource. Insuch scenarios, the stream pacer 136 allows a tuple to execute only whenall or a predefined number of the constraints are satisfied by therespective tuple.

Further, in one embodiment, the stream pacer 136 also evaluates tuplesthat are currently being processed, against the maximum allowedprocessing duration, to determine if the processing of any of the tuplesshould be halted. Although the stream pacer 136 may previously haveallowed the tuples to be processed, the system load of the processingelement 235 may since have increased to such an extent as to warranthalting the previously allowed tuples mid-processing. For example,suppose that the stream pacer 136 approves, for execution, a tupleestimated to take six seconds to execute, because the tuple does notexceed a maximum allowed processing duration of eight seconds. Supposethat three seconds after approving the tuple for execution, the streampacer 136 determines a new maximum processing duration of two hundredmilliseconds, as a result of a sudden increase in system load of theprocessing element 235. Because the remaining processing duration of thetuple (i.e., three seconds) exceeds the new maximum processing duration,the stream pacer 136 may halt the processing of the tuple. As describedabove, in some embodiments, it may not be desirable to halt theprocessing of tuples that are being processed. Accordingly, a user mayset a flag that specifies not to halt processing tuples under anycircumstances. Thus, the behavior of the stream pacer 136 in haltingtuple processing may be overridden.

In some embodiments, rather than merely operating on a single tuple, thestream pacer 136 may also operate on a group of tuples. In suchembodiments, the stream pacer 136 may approve or reject the entire groupfor processing, based on a comparison between a maximum duration forwhich a given processing element is allowed to process the group and anestimated duration for which the processing element is likely to processthe group. Each group may be identified based on predefined criteria,such as based on the requesting entity, incoming tuple rates, windowingconditions, data attributes stored in the tuples, etc. Windowingconditions are a set of criteria which determine tuples within aspecified group that are being operated on at a given time. For example,a time-based windowing condition for an associated processing operationmay state that only tuples generated in the last ten seconds areconsidered to be within the scope of a window for the associatedprocessing operation. The condition may be based on a number of factors,including but not limited to any specified time duration and/or tuplecount, etc. At least some of the predefined criteria may be provided bythe buffered stream monitor 245 associated with the processing element.Depending on the embodiment, the group of tuples may also spanprocessing elements. In such embodiments, a single determination may bemade of whether the group of tuples may be executed—regardless of whichprocessing element is to execute each particular tuple in the group.

In some embodiments, instead of determining maximum and/or estimateddurations relative to a single processing element 235, the stream pacer136 may also determine maximum and/or estimated durations relative to aplurality of processing elements 235. The plurality of processingelements may correspond to a predefined section of the operator graph132. In some embodiments, the plurality of processing elements mayselected by a user. In other embodiments, the plurality of processingelements may be identified based on user-specified criteria, such asrequesting entity, incoming tuple rates, windowing conditions, dataattributes stored in the tuples, etc.

FIG. 7 is a flowchart depicting a method 700 for evaluating a tuple forprocessing, according to one embodiment of the invention. As shown, themethod 700 begins at step 710, where the stream pacer 136 identifies atuple from a requesting entity, for execution by a processing element.For example, the stream pacer 136 may identify the first tuple of FIG.2C. At step 320, the stream pacer 136 determines a maximum duration forwhich the processing element is allowed to process the tuple. Themaximum duration may be determined based on a measure of current load ofthe processing element, such as an indication of a current buffer levelassociated with the processing element. For instance, the stream pacer136 may determine, based on a low buffer level, that the maximumprocessing duration should be eight hundred milliseconds.

At step 730, the stream pacer 136 determines an estimated duration,where the estimated duration refers to a duration for which theprocessing element is likely to process the tuple. For example, thestream pacer 136 may determine that the processing element is likely totake seven hundred milliseconds to process the tuple, as represented bythe bar 602 in FIG. 6C. At step 740, the stream pacer 136 managesprocessing of the tuple, based on a comparison between the maximumduration and the estimated duration. The step 740 is further describedbelow with reference to the method 800 of FIG. 8. After the step 740,the method 700 terminates.

FIG. 8 is a flowchart depicting a method 800 for managing processing ofthe tuple, according to one embodiment of the invention. The method 800corresponds to the step 740 of FIG. 7. As shown, the method 800 beginsat step 810, where the stream pacer 136 determines whether the estimatedduration exceeds the maximum duration, where the estimated and maximumdurations are computed in steps 720 and 730 of FIG. 7, respectively. Ifthe estimated duration does not exceed the maximum duration, then thestream pacer 136 allows the tuple to be processed by the processingelement (step 820). On the other hand, if the estimated duration exceedsthe maximum duration, then the stream pacer 136 determines whether thetuple is currently being processed (step 830). If so, then the streampacer 136 may halt the processing of the tuple.

Alternatively, if the tuple is not currently being processed (step 830),then the stream pacer 136 may reject the tuple from being processed bythe processing element (step 850). At least in some embodiments, thestream pacer 136 may also allow a different processing element toprocess the rejected tuple. Alternatively, the stream pacer 136 maydelay processing of the tuple until the load experienced by theprocessing element is reassessed, by which point the load may havebecome light enough as to allow execution of the tuple by the processingelement. At least in some embodiments, the stream pacer 136 assesses theload according to a predefined schedule. For example, the load may beassessed in five-minute intervals. A user may tailor the schedule tosuit the needs of a particular case. The stream pacer 136 may alsoreturn a message to a requesting entity, explaining that execution ofthe tuple is denied and/or delayed. After the step 820, the step 840, orthe step 850, the method 800 terminates.

Advantageously, embodiments of the invention provide techniques forevaluating a tuple for processing. One embodiment provides a streampacer that is configured to identify a tuple to be processed by aprocessing element in a steam application. The stream pacer determines amaximum duration for which the processing element is allowed to processthe tuple. The stream pacer also determines an estimated duration forwhich the processing element is likely to process the tuple. The streampacer then manages processing of the tuple, based on a comparisonbetween the maximum duration and the estimated duration. Advantageously,the stream pacer may manage tuple processing in a manner that is moreresponsive to the needs of processing elements in and/or users of thestream application.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

What is claimed is:
 1. A computer program product, comprising: acomputer-readable memory having computer-readable program code embodiedtherewith, the computer-readable memory including hardware, thecomputer-readable program code comprising: computer-readable programcode configured to receive streaming data to be processed by a streamapplication comprising a plurality of processing elements, wherein eachprocessing element includes one or more operators that are containedwithin a single process, each operator having executable code configuredto connect to one or more other operators, and wherein the streamingdata includes a plurality of tuples to be processed by differentprocessing elements of the stream application, each tuple having one ormore attributes; computer-readable program code configured to provide anoperator graph of processing elements, the operator graph defining atleast one execution path for processing the received streaming data, atleast a first processing element of the operator graph configured toreceive data from at least one upstream processing element and transmitdata to at least one downstream processing element; computer-readableprogram code configured to identify at least one tuple to be processedby at least one processing element of the stream application;computer-readable program code configured to determine a maximumduration for which the at least one processing element is allowed toprocess the at least one tuple; computer-readable program codeconfigured to determine an estimated duration for which the at least oneprocessing element is likely to process the at least one tuple; andcomputer-readable program code configured to manage processing of the atleast one tuple, based on a comparison between the maximum duration andthe estimated duration.
 2. The computer program product of claim 1,wherein the maximum duration is determined based on at least one of userinput and a workload experienced by the at least one processing element.3. The computer program product of claim 1, wherein the estimatedduration is determined based on historical usage data collected fromtuples previously processed by the at least one processing element. 4.The computer program product of claim 1, wherein the computer-readableprogram code configured to manage execution of the tuple comprises:computer-readable program code configured to approve the at least onetuple to be processed by the at least one processing element, upondetermining that the estimated duration does not exceed the maximumduration.
 5. The computer program product of claim 1, wherein the atleast one processing element includes a first processing element, andwherein the computer-readable program code configured to manageprocessing of the at least one tuple comprises computer-readable programcode configured to, upon determining that the estimated duration exceedsthe maximum duration, perform at least one of: rejecting the at leastone tuple from being processed by the first processing element; andapproving the at least one tuple to be processed by a second processingelement of the stream application.
 6. The computer program product ofclaim 1, wherein the computer-readable program code further comprisescomputer-readable program code configured to approve the at least onetuple for processing by the at least one processing element, and whereinthe computer-readable program code configured to manage processing ofthe at least one tuple further comprises: computer-readable program codeconfigured to halt the processing of the at least one tuple by the atleast one processing element, upon determining that the estimatedduration exceeds the maximum duration.
 7. A system, comprising: one ormore computer processors; a memory containing a program, which whenexecuted by the one or more computer processors is configured to performan operation comprising: receiving streaming data to be processed by astream application, wherein the stream application comprises a pluralityof processing elements, each processing element including one or moreoperators that are contained within a single process, each operatorhaving executable code configured to connect to one or more otheroperators, and wherein the streaming data includes a plurality of tuplesto be processed by different processing elements of the streamapplication, each tuple having one or more attributes; providing anoperator graph of processing elements, the operator graph defining atleast one execution path for processing the received streaming data, atleast a first processing element of the operator graph configured toreceive data from at least one upstream processing element and transmitdata to at least one downstream processing element; identifying at leastone tuple to be processed by at least one processing element;determining a maximum duration for which the at least one processingelement is allowed to process the at least one tuple; determining anestimated duration for which the at least one processing element islikely to process the at least one tuple; and managing processing of theat least one tuple, based on a comparison between the maximum durationand the estimated duration.
 8. The system of claim 7, wherein themaximum duration is determined based on at least one of user input and aworkload experienced by the at least one processing element.
 9. Thesystem of claim 7, wherein the estimated duration is determined based onhistorical usage data collected from tuples previously processed by theat least one processing element.
 10. The system of claim 7, whereinmanaging execution of the at least one tuple comprises: approving the atleast one tuple to be processed by the at least one processing element,upon determining that the estimated duration does not exceed the maximumduration.
 11. The system of claim 7, wherein the at least one processingelement includes a first processing element, and wherein managingprocessing of the at least one tuple comprises, upon determining thatthe estimated duration exceeds the maximum duration, at least one of:rejecting the at least one tuple from being processed by the firstprocessing element; and approving the at least one tuple to be processedby a second processing element of the stream application.
 12. The systemof claim 7, wherein the maximum duration is determined based on at leastone of user input and a workload experienced by the at least oneprocessing element.
 13. The system of claim 12, wherein the estimatedduration is determined based on historical usage data collected fromtuples previously processed by the at least one processing element. 14.The system of claim 13, wherein managing execution of the at least onetuple comprises: approving the at least one tuple to be processed by theat least one processing element, upon determining that the estimatedduration does not exceed the maximum duration.
 15. The system of claim14, wherein the at least one processing element includes a firstprocessing element, and wherein managing processing of the at least onetuple comprises, upon determining that the estimated duration exceedsthe maximum duration, at least one of: rejecting the at least one tuplefrom being processed by the first processing element; and approving theat least one tuple to be processed by a second processing element of thestream application.
 16. The system of claim 15, wherein the at least oneprocessing element includes a first processing element, wherein managingprocessing of the at least one tuple in a first instance comprises, upondetermining that the estimated duration exceeds the maximum duration,rejecting the at least one tuple from being processed by the firstprocessing element.
 17. The system of claim 16, wherein managingprocessing of the at least one tuple in a second instance comprises,upon determining that the estimated duration exceeds the maximumduration, approving the at least one tuple to be processed by a secondprocessing element of the stream application.
 18. The system of claim17, wherein managing processing of the at least one tuple in a thirdinstance comprises, upon determining that the estimated duration exceedsthe maximum duration: rejecting the at least one tuple from beingprocessed by the first processing element; and approving the at leastone tuple to be processed by a second processing element of the streamapplication.